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0 A re-edit function allowing bi-directional rippling. 
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© A computer controlled video re-editing system 
with a "Domino" re-edit function. An edited video 
comprises a set of ordered video segments. In a 
video re-editing system a user can re-edit an edited 
video by adjusting the duration of each video seg- 
ment When a user adjusts the length of a video 
segment, the Domino re-edit function examines how 
the re-edit to the edited video work affects nearby 
video segments in the edited video work. The 
Domino re-edit function then propagates the effects 
of a re-edit by moving the nearby video segments 
forward or backward along a time line to accom- 
modate the re-edit made. The Domino re-edit func- 
tion utilizes any empty spaces that exist in an edited 
video work such that the empty spaces are filled 
before the effects of a re-edit are propagated to 
another video segment. If there is sufficient empty 



space between video segments, the effects of the 
re-edit cease propagating. To prevent a video seg- 
ment from being moved by the Domino re-edit func- 
tion, a user can "lock" the video segment to a 
reference time line. To preserve a relative relation- 
ship between two video segments, a user can lock 
the two video segments together. 



] [ 



Figure 4a 
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The present invention relates to apparatus and 
methods for editing video and audio information, 
and more particularly, the present invention relates 
to a computer controlled re-editing system which 
fills in gaps in a media time line when a re-edit is 
made. 

Video editing combines video source material 
from a number of different video sources into a 
single video work. After the editing process, an 
edited video work comprises an ordered collection 
of video blocks from the different video sources. 
Each video block comprises an "in" point which 
marks where the video block begins and an or 
"out" point which designates where the video block 
ends. 

Edited video works are then usually "re-edited" 
to further refine the edited video work. During the 
re-editing process the existing collection of video 
blocks may be moved around. Furthermore, the in 
and out points of each video block may be moved 
forward (ascending in time) or backward (descend- 
ing in time). Keeping track of and adjusting all the 
in and out points of the video blocks during the re- 
edit process is a computationally intensive task. 
Consequently, computer controlled video re-editing 
systems have been developed. The computer con- 
trolled video re-editing system manages the task of 
adjusting the in and out points which define the 
video blocks as a user re-edits an edited video 
work. 

One problem associated with existing computer 
controlled video re-editing systems is that the re- 
editing systems only adjust the video block in and 
out points after re-edits in one of two simple ways. 
The existing re-editing systems either affect only 
the single video block being re-edited or propagate 
the effects of the re-edit to every subsequent video 
block. Such all or nothing re-edit systems lack the 
flexibility needed for today's heavily edited video 
productions. As will be described, the present in- 
vention provides a method and apparatus for re- 
editing video such that a re-edit to a video block 
only affects the nearby video blocks as necessary 
to accommodate the re-edit. 

It is therefore an aim of the present invention to 
provide a computer controlled re-edit system with 
more flexible re-edit operations. Specifically, the 
present invention implements a "Domino" re-edit 
function that adjusts only the nearby video blocks 
as necessary to accommodate a re-edit. 

The Domino re-edit function of the present 
invention first examines the video blocks which are 
immediately affected by a re-edit made to an ed- 
ited video work. The Domino re-edit function then 
propagates the effects of a re-edit by moving the 
nearby video blocks forward or backward along a 
time line to accommodate the re-edit made. The 
Domino re-edit function utilizes any empty spaces 



that exist in an edited video work such that the 
empty spaces are filled before the effects of a re- 
edit are propagated to another video block. If there 
is sufficient empty space between video blocks, the 

5 effects of the re-edit cease propagating. To prevent 
a video block from being moved by the Domino re- 
edit function, a user can "lock" the video block to a 
reference time line. 

The Domino re-edit function of the present 

10 invention is implemented on a computer controlled 
re-editing system. The computer controlled re-ed- 
iting system comprises at least one central pro- 
cessing unit (CPU) is coupled through appropriate 
input/output (I/O) circuitry to input devices, such as 

is a cursor control device. The CPU is further coupled 
to a storage device for the storage of programs and 
data including a database storing information about 
all the source video material. The CPU is also 
coupled to a display device on which a graphical 

20 user interface is displayed. 

The features, and advantages of the present 
invention will be apparent from the following de- 
tailed description of an exemplary embodiment of 
the invention with references to the drawings in 

25 which: 

Figure 1 is a functional block diagram illustrat- 
ing one possible computer display system incor- 
porating the teachings of the present invention. 
Figure 2 is a conceptual diagram of video sour- 
30 ces and an edited video made from the video 
sources. 

Figure 3a illustrates a typical re-edit on an 
edited video where the out point of a video 
block is moved forward in time. 
35 Figure 3b illustrates the reconciliation of the re- 
edit illustrated in Figure 3a by a prior art re-edit 
system configured to confine the re-edit to the 
re-edited video block and the affected video 
block. 

40 Figure 3c illustrates the reconciliation of the re- 
edit illustrated in Figure 3a by a prior art re-edit 
system configured to propagate the re-edit to all 
the video blocks in the edited video. 
Figure 4a illustrates the reconciliation of the re- 

45 edit illustrated in Figure 3a by the Domino 
function which moves the affected video blocks 
only as necessary. 

Figure 4b illustrates the reconciliation of a re- 
edit by rippling the re-edit to subsequent video 

50 blocks as necessary. 

Figure 5a illustrates the reconciliation of a re- 
edit by the Domino function when a user has 
requested the effects be confined to a contig- 
uous stet of video blocks. 

55 Figure 5b illustrates the reconciliation of a re- 
edit by the Domino function when a user has 
requested the effects of the re-edit be confined 
to a contiguous set of video blocks but the re- 
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edit requires more space than is available. 
Figure 6 illustrates a typical use of locking a 
video block to a reference time line. 
Figure 7 illustrates the reconciliation of a re-edit 
by the Domino function when a video block has 
been locked to the reference time line and the 
re-edit cannot be performed since the locked 
video block will not move. 

Figure 8a illustrates the use of a relative lock 
connecting two adjacent video blocks. 
Figure 8b illustrates a use of a relative lock 
connecting two video blocks together while re- 
serving space between the video blocks. 
Figure 9 illustrates the reconciliation of a re-edit 
by the Domino function when two video blocks 
have been locked reserving space between the 
video blocks such that the Domino re-edit main- 
tains the space between the two locked video 
blocks. 

Figure 10 illustrates the reconciliation of a re- 
edit by the Domino function when two video 
blocks have been locked to associated audio 
tracks. 

Figure 11 illustrates a backward (descending in 

time) Domino ripple re-edit. 

Figure 12 illustrates a Domino ripple re-edit 

which requires both a forward and backward 

ripple. 

NOTATION AND NOMENCLATURE 

The detailed descriptions which follow are pre- 
sented largely in terms of display images, algo- 
rithms, and symbolic representations of operations 
of data bits within a computer memory. These 
algorithmic descriptions and representations are 
the means used by those skilled in the data pro- 
cessing arts to most effectively convey the sub- 
stance of their work to others skilled in the art. 

An algorithm is here, and generally, conceived 
to be a self consistent sequence of steps leading to 
a desired result. These steps are those requiring 
physical manipulations of physical quantities. Usu- 
ally, though not necessarily, these quantities take 
the form of electrical or magnetic signals capable 
of being stored, transferred, combined, compared, 
and otherwise manipulated. It proves convenient at 
times, principally for reasons of common usage, to 
refer to these signals as bits, values, elements, 
symbols, characters, images, terms, numbers, or 
the like. It should be borne in mind, however, that 
all of these and similar terms are to be associated 
with the appropriate physical quantities and are 
merely convenient labels applied to these quan- 
tities. 

In the present case, the operations are ma- 
chine operations performed in conjunction, with a 
human operator. Useful machines for performing 



the operations of the present invention include gen- 
eral purpose digital computers or other similar de- 
vices. In all cases, there should be borne in mind 
the distinction between the method operations of 

5 operating a computer and the method of computa- 
tion itself. The present invention relates to method 
steps for operating a computer and processing 
electrical or other physical signals to generate oth- 
er desired physical signals. 

io The present invention also relates to apparatus 

for performing these operations. This apparatus 
may be specially constructed for the required pur- 
poses, or it may comprise a general purpose com- 
puter selectively activated or reconfigured by a 

75 computer program stored in the computer. The 
algorithms, methods and apparatus presented 
herein are not inherently related to any particular 
computer. In particular, various general purpose 
machines may be used with programs in accor- 

20 dance with the teachings herein, or it may prove 
more convenient to construct more specialized ap- 
paratus to perform the required method steps. The 
required structure for a variety of these machines 
will appear from the description given below. Ma- 

25 chines which may perform the functions of the 
present invention include those manufactured by 
Sony Corporation of America, as well as other 
manufacturers of Computer systems. 

30 DETAILED DESCRIPTION OF THE INVENTION 

The present invention discloses apparatus and 
methods for a computer controlled editing system 
that has a re-edit function with bi-directional rip- 

35 pling. In the following description, numerous spe- 
cific details are set forth such as computer system 
configurations, window elements, icons, desktop 
sizes, metaphors, window configurations and ar- 
rangements, etc. in order to provide a thorough 

40 understanding of the present invention. However, it 
will be apparent to one skilled in the art that the 
present invention may be practiced without these 
specific details. In other instances, well known cir- 
cuits, structures and the like are not described in 

45 detail so as not to obscure the present invention 
unnecessarily. 

Hardware Configuration 

so Referring now to Figure 1, the hardware con- 

figuration of the computer controlled re-edit system 
of the present invention is illustrated in block dia- 
gram form. The figure illustrates a computer sys- 
tem for generating a window-based graphic user 

55 interface coupled to control hardware for controlling 
resources that store source material. As illustrated, 
the re-edit system includes a computer 20 which 
comprises four major components. The first of 
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these is an input/output (I/O) circuit 22, which is 
used to communicate information in appropriately 
structured form to and from other portions of the 
computer 20. In addition, computer 20 includes a 
central processing unit (CPU) 24 coupled to the I/O 
circuit 22 and to a memory 26. These elements are 
those typically found in most computers and, in 
fact, computer 20 is intended to be representative 
of a broad category of data processing devices. 

As illustrated in Figure 1 a keyboard 30 is 
coupled to computer 20 through the I/O circuit 22 
for inputting data and commands, as is well known 
in the art. Also coupled to the I/O circuit 22 is a 
mass memory system 32 for storing programs and 
data. Similarly, a CD ROM 34 is coupled to the I/O 
circuit 22 for providing additional data storage ca- 
pacity to the system illustrated in Figure 1. It will 
be appreciated that other common computer pe- 
ripherals may be coupled to the computer 20 such 
as magnetic tape drives and a computer network 
interface. 

A device control interface 36 is coupled to both 
the memory 26 and the I/O circuit 22, to permit the 
computer 20 to communicate with and control me- 
dia resources. For example, in a video editing 
environment, the computer 20 may be coupled to a 
video tape recorder 40, a music synthesizer 42, an 
audio tape recorder 44, and, special effects re- 
source 46, as shown in Figure 1. 

A display monitor 50 is coupled to the com- 
puter 20 through the I/O circuit 22. Any well known 
variety of cathode ray tube (CRT), liquid crystal or 
other displays may be utilized as display 50. A 
cursor control device 52 includes stitches 54 and 
56 for signally the CPU 24 in accordance with the 
teachings of the present invention. Cursor control 
device 52 (commonly referred to a "mouse") per- 
mits a user to select various command modes, 
modify graphic data, and input other data utilizing 
switches 56 and 54. More particularly, the cursor 
control device 52 permits a user to selectively 
position a cursor 58 at any desired location on a 
display screen 60 of the display 50. As illustrated 
in Figure 1, the cursor 58 is disposed with a 
window 65 in the present invention's graphic user 
interface, to be described more fully below. More- 
over, in the presently preferred embodiment, the 
present invention's window-based user interface is 
generated and displayed using software stored in 
either memories 26, 32 or CD ROM 34, and ex- 
ecuted by the CPU 24. 

In the presently preferred embodiment, cursor 
control 52 utilizes well known apparatus and meth- 
ods for signaling CPU 24 of positional changes of 
cursor 58 by movement of the cursor control over a 
surface. However, it will be appreciated by one 
skilled in the art that a variety of well known cursor 
control devices may be utilized by the present 



invention, including other control devices such as 
mechanical mice, trackballs, joy sticks, graphic tab- 
lets, other keyboard inputs and the like. The cursor 
control 52 in Figure 1 is intended to encompass all 
5 such equivalent devices. 

Video Editing 

The first step in the video editing process is to 

10 catalog all of the source material that has been 
created such that a database of the available 
source material is created. For example, a com- 
puter controlled editing system may provide a 
means for a user of the computer controlled editing 

75 system to generate source logs. The source logs 
comprise selected portions of audio and/or video 
generated from the various media resources. Each 
source log comprises information such as a tape 
identifier, an start point, and end point, and any 

20 other useful information that will be used during the 
editing and re-editing processes. The source logs 
define the source material for use in the computer 
controlled editing system that allows a user to 
select the source material for editing or re-editing 

25 of a production. To create a source log, a user may 
select a media resource, such as a video tape 
player, and command the computer controlled ed- 
iting system to capture a selected portion of the 
video generated by the video tape player. The 

30 selected portion of the video captured constitutes a 
source log. 

The source log generated may comprise video 
and/or any number of audio channels. The plurality 
of source logs generated for editing of a particular 

35 multi-media production may be arranged into a file 
such that the file contains source logs for the 
editing or re-editing of that particular production. As 
will be described more fully below, the new 
Domino re-editing function of the present invention 

40 operates in conjunction with the source material 
database comprising source logs. Although the 
present invention is described in conjunction with a 
computer controlled editing system that generates 
source logs, any database comprising source ma- 

45 terial for use in a computer controlled editing sys- 
tem could be used in accordance with the teach- 
ings of the present invention. 

Figure 2 illustrates video source material being 
used to construct an edited video. On the left side 

so of Figure 2 there are three sections of video 
source material. Each section of video source ma- 
terial has a start point and an end point. Between 
the start point and end point are a plurality of video 
frames. The video source material is cataloged into 

55 a database as described above such that the 
database contains a tape identifier, a start point, an 
end point, and other useful information that will be 
used during the editing and re-editing processes. 
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To edit the video source material, an operator 
using a computer controlled editing system builds 
a list of in and out points which represent sub- 
sections of the video available in the video source 
material. Each in point and out point pair defines a 
video block such as video blocks A, B, C, and D. 
After the editing process a the final product is the 
collection of ordered video blocks as illustrated on 
the right side of Figure 2. The ordered video 
blocks are laid out along a time line that provides a 
time reference. 

Referring to Figure 2, gaps exists between 
some of the video blocks in the edited video work. 
Specifically, there are empty spaces between video 
blocks B and C and between video blocks C and 
D. These empty spaces represent time where there 
is no video output. The empty spaces are often 
called "implicit black" since empty black video 
frames will be output during those times unless the 
empty spaces are eventually filled with other video 
source material. 

Although Figure 2 illustrates only a single 
track, an editing system can operate on multiple 
tracks such that the final work comprises any num- 
ber of different media tracks. Currently, the most 
editing systems operate on a single video track 
with two accompanying audio tracks. For simplicity, 
this Specification will usually only discuss oper- 
ations on a single track. However, the present 
invention is meant to be used in re-editing systems 
that operate on multiple tracks. 

Prior Art Re-Editing 

Re-editing consists of the process of further 
refining an edited video work. During the re-editing 
process the existing collection of video blocks may 
be moved around. Furthermore, the in and out 
points defining each video block may be adjusted 
by being moved forward (ascending in time) or 
moved backward (descending in time). 

For example, Figure 3a illustrates a simple re- 
edit of the edited video illustrated in Figure 2. In 
the re-edit illustrated in Figure 3a, the out point of 
video block B has been moved forward in time 
thereby extending the duration of video block B. As 
graphically illustrated in Figure 3a, the out point of 
video block B extends past the in point of video 
block C. The extension creates an inconsistent 
time line in the edited video since video blocks B 
and C overlap, Therefore the re-editing system 
must somehow reconcile the re-edit made to video 
block B in order to maintain a consistent time fine. 

Prior art re-editing systems are severely limited 
in the ways such systems can reconcile re-edits 
that create inconsistent time lines. Generally, prior 
art re-editing systems reconcile inconsistent time 
lines only in one of two ways. A first method of 
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reconciling the time line is to modify only the in or 
out point of the adjacent video block. Alternatively, 
prior art re-edit systems reconcile inconsistent time 
lines by modifying every subsequent video block 

5 such that the re-edit affects all the video blocks 
located after the re-edited video block. 

Examining the first method, when the re-edit 
illustrated in Figure 3a is reconciled by a prior art 
re-edit system configured to modify only the adja- 

io cent in or out point, then only the in point of video 
block C is modified. Figure 3b illustrates the. ed- 
ited video time line after the in point of video block 
C is modified. In Figure 3b the in point of video 
block C has been moved forward in time such that 

75 video block B no longer overlaps video block C. 
This method has the disadvantage of shortening 
the duration video block C as illustrated in Figure 
3b. 

In the second method, when the re-edit illus- 
20 trated in Figure 3a is reconciled by a prior art re- 
edit system that is configured to modify every 
subsequent video block, all the video blocks lo- 
cated after video block B must be moved back in 
the same manner as the out point for video block 
25 B. Figure 3c illustrates the edited video time line 
after every video block after video block B has 
been moved back. 

As illustratecfgraphically in Figures 3b and 3c, 
prior art re-edit systems do not recognize or utilize 
30 the empty spaces between video blocks. In many 
situations it is be desirable to have a more flexible 
re-editing system that would use these empty 
spaces. Therefore the present invention creates a 
number of different ways of handling re-edits. 

35 

The Domino Re-Edit Function 

The present invention introduces a new type of 
re-edit function entitled the "Domino" re-edit. A 

40 Domino re-edit is performed when a user wishes to 
extend a video block and move over all the subse- 
quent video blocks only as necessary to make 
room for the extension. Since the Domino re-edit 
moves video blocks only as necessary, the Domino 

45 re-edit differs from the prior re-edit propagation of 
Figure 3c since the Domino re-edit fills in empty 
spaces available in the edited video time line. Thus 
a Domino re-edit is conceptually similar to a series 
of dominoes standing on end which knock over the 

so adjacent domino unless there is sufficient space 
between the dominoes. 

For example, referring to Figure 4a, a Domino 
re-edit is illustrated. The first video time line of 
Figure' 4a illustrates the edited video from Figure 

55 2. The second video time line of Figure 4a illus- 
trates the re-edit made in Figure 3a. The third 
video time line of Figure 4a illustrates how the 
Domino re-edit reconciles the inconsistent time line 

5 
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created by the re-edit in the second video time 
line. In the third video time line of Figure 4a, video 
block C has moved over to the right filling in the 
gap that previously existed between video block C 
and video block D. Since there was enough empty 
space in the gaps between the video blocks, video 
block D was unaffected by the re-edit. 

Figure 4b illustrates a Domino re-edit that "rip- 
ples" the re-edit to other video blocks. The first 
video time line of Figure 4b illustrates* the edited 
video from Figure 2 with a fifth video block added. 
The second video time line of Figure 4b illustrates 
a re-edit where the out point of video block B 
extends well into video block C. The bottom video 
time line of Figure 4b illustrates how the Domino 
re-edit reconciles the inconsistent time line created 
by the re-edit in the second video time line. In the 
bottom video time line of Figure 4a, video block C 
has moved over to the right covering the gap that 
previously existed between video block C and vid- 
eo block D. Since there was not enough empty 
space in the gaps between video blocks C and D, 
the system also had to ripple the re-edit to video 
blocks D and E by moving video blocks D and E 
forward in time. 

Thus the Domino re-edit function of the present 
invention utilizes the empty spaces between video 
blocks when a re-edit is performed. The Domino 
re-edit function ripples the effects of a re-edit to 
other video blocks only when there are not enough 
empty spaces between the video blocks to accom- 
modate the re-edit. 

Domino Re-Edit isolated to Contiguous Seg- 
ment 

In some situations, a user of a re-edit system 
may wish to confine the affects of a re-edit to a 
particular contiguous segment. For example, if the 
user is re-editing only one particular scene in a 
video program, the user may want other scenes of 
the program to remain unaffected by the re-edits. 
To accommodate this situation, the present inven- 
tion provides a mode where the user can specify 
that the re-edits only affect a contiguous set of 
video blocks. 

Figure 5a illustrates a first example of a re-edit 
confined to a contiguous set of video blocks. The 
first video time line illustrated in Figure 5a has 
three different sets of contiguous video blocks. On 
the second video time line illustrated in Figure 5a 
the out point of video block C in the second 
contiguous video block has been extended past the 
in point of video block D also in the second contig- 
uous video block. The third video time line of 
Figure 5a illustrates how the Domino re-edit recon- 
ciles the inconsistent time line created by the re- 
edit in the second video time line. In the third video 



time line of Figure 5a, video blocks D and E are 
moved over to the right. Since there was enough 
empty space between the second contiguous video 
block and the third contiguous video block, only the 
5 second set of contiguous video blocks was af- 
fected. 

Figure 5b illustrates a second example of a re- 
edit confined to a contiguous set of video blocks. 
The first video time line illustrates the same three 

70 sets of contiguous video blocks from Figure 5a. 
On the second video time line illustrated in Figure 
5b the out point of video block C in the second 
contiguous video block has been extended far past 
the in point of video block D. The third video time 

75 line of Figure 5b illustrates how the Domino re-edit 
in system attempts to reconcile the inconsistent 
time line created by the re-edit in the second video 
time line. In the third video time line of Figure 5b, 
video blocks D and E are moved over to the right. 

20 However, since there was insufficient empty space 
between the second contiguous video block and 
the third contiguous video block, an overlap exists 
between video block E in the second contiguous 
video block and video block F in the third contig- 

25 uous video block. The re-edit system of the 
present invention recognizes and handles the over- 
lap problem. 

In one embodiment, the present invention dis- 
plays a pop-up dialog box which explains the over- 

30 lap problem to the user. The system asks the user 
whether the re-edit should aborted or whether the 
system should move the third set of contiguous 
video blocks forward in time. 

In a second embodiment, the present invention 

35 uses a set of predefined user preferences to han- 
dle the overlap problem. For example, the user 
may specify that the system always abort any re- 
edit which causes an overlap problem. 

40 Domino Re-Edits With Absolute Locks 

Generally, the Domino re-edit function moves 
video blocks as necessary to reconcile re-edits. In 
certain situations a user may require that a particu- 

45 lar video block remain at a particular position on 
the time line. For example, when an operator uses 
the present invention to edit a music video, the 
operator may want to synchronize certain video 
blocks with the audio track. To accommodate such 

so situations, the present invention provides a method 
of "locking" video blocks to a time line. When a 
video block is locked to the time line, the Domino 
re-edit function will not ever move the locked video 
block wh6n reconciling a re-edit. 

55 Referring to Figure 6 a typical use of locked 

video blocks is illustrated. At the bottom of Figure 
6 a time line is illustrated. The time line provides a 
time reference upon which the media tracks are 
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placed. Above the reference time line is an audio 
track which has been laid on the time line. In the 
example illustrated in Figure 6, the audio track 
comprises the song for which a music video is 
being created. The audio track has been locked to 
the reference time line such that song will not 
move. Near the beginning of the song, the first 
verse of the song begins. A block of video contain- 
ing shots of the lead singer singing the first verse 
of the song has been placed at the point where the 
first verse of the song begins such that the video 
block of the singer is synchronized with the song. 
The video block of the singer is then locked to the 
reference time line such that the video block will 
net be affected by any further edits. Similarly, near 
the end of the song at point y is a loud cymbal 
crash To complement the loud cymbal crash at 
point y. the video director has locked a video block 
containing a dramatic explosion. The locked dra- 
m.itK explosion video block also will not be af- 
i.vt.-i hy any further re-edits. 

Figure 7 illustrates a first example of a re-edit 
f»-fii-*m»Ki on an edited video containing a video 
t » <b that is locked to the time line. The first video 
t.rr>« line- .iiustrated in Figure 7 has four video 
ttfc.;hs A. B C. and D. The fourth video block, 
v v-* Diock D. is locked to the reference time line 
to prevent video block D from moving. In the 
s*. :L>nd video time line illustrated in Figure 7, the 
oct comt of video block A has been extended past 
tn- in point of video block B such that the re-edit 
system must reconcile the time line. The third 
vk**- time line of Figure 7 illustrates how the 
system reconciles the inconsistent time line. In the 
third viaeo time line of Figure 7, the system has 
moved video blocks B and C to the right after 
e-terded video block A, The system does not 
move Video block D since Video block D has been 
locked to the reference time line. Since there was 
net enough empty space between the video blocks, 
videc block C overlaps locked video block D. The 
rc-edit system of the present invention recognizes 
ard handles the overlap problem. 

In one embodiment, the present invention dis- 
plays a pop-up dialog box which explains the over- 
lap pobiem to the user. The dialog box requests 
the user whether the system should abort the re- 
eot or move locked video block D forward in time 
to accommodate the edit. 

In a second embodiment, the present invention 
uses a set of predefined user preferences to han- 
dle the overlap problem. For example, the user 
may specify that the system always abort any re- 
eoit which causes an overlap problem. 



Domino Re-Edits With Relative Locks 

An operator editing a video often does not 
know exacting where on a reference time line a 

5 particular video block should be placed. However, 
the operator may know other information such as 
the relative relationships between different video 
shots. The concept of a "locked" video block can 
be used to lock various relationships between 

io shots. 

For example, in a movie containing a car bomb 
explosion an operator may want to have a scene of 
the ignition key being turned immediately followed 
by an explosion. To permanently lock this relation- 
75 ship between these two video shots, a user of the 
present invention can place a "relative lock" con- 
necting these two video shots together as illus- 
trated in Figure 8a. 

Re-editing a video with missing material pro- 
20 vides another good example of a use for a relative 
lock. Figure 8b illustrates an edited video without 
all the required video source material. The video 
begins with a video block of astronauts boarding a 
rocket. The next scene should contain a special 
25 effects sequence of the rocket blasting off. How- 
ever, the blast off scene has not yet been taped so 
there is an empty gap in the edited video. After the 
blast off scene iS a video block of the astronauts 
grimacing and sweating during the rocket launch. 
30 To reserve space on the time line for the blast off 
scene, a relative lock connecting the first video 
block to the second video block with an empty 
space in-between has been added. 

The re-edit system of the present invention 
35 honors the relative locks by always keeping the 
locked video blocks in the locked relative relation- 
ship even though the locked video blocks may 
move along the reference time line. Figure 9 illus- 
trates an example of this effect. 
40 Figure 9 illustrates an example of a re-edit 

performed on an edited video that has two video 
blocks locked to one another in a relative relation- 
ship. The first video time line illustrated in Figure 9 
has five video blocks A, B, C, D, and E. The 
45 second and third video blocks, video blocks B and 
C, are locked together in a relative relationship with 
empty space between the two video blocks similar 
to the video blocks illustrated in Figure 8b. In the 
second video time line illustrated in Figure 9, the 
so out point of video block A has been extended past 
the in point of video block B such that the system 
must reconcile the time line. The third video time 
line of Figure 9 illustrates how the system recon- 
ciles the inconsistent video time line which contains 
55 a relative lock. In the third video time line of Figure 
9, the system has moved video blocks B, C, D, 
and E forward in time after extended video block A. 
The system moves locked video blocks B and C as 
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a single unit such that the space between the two 
video blocks has been preserved. 

Relative locks can also be used to lock blocks 
of different media tracks together. Figure 10 illus- 
trates an example of a re-edit performed on an 
edited video with audio where two video blocks are 
locked to associated audio tracks. The first video 
time line illustrated in Figure 10 has four video 
blocks A, B, C, and D. Video blocks B and D have 
associated audio tracks B* and D' which are locked 
to video blocks B and D respectively. In the sec- 
ond video time line illustrated in Figure 10, the out 
point of video block A has been extended past the 
in point of video block B such that the system must 
reconcile the time line. The third video time line of 
Figure 10 illustrates how the system reconciles the 
inconsistent video time line which contains locked 
audio tracks. In the third video time line of Figure 
10, the system has moved video blocks B and C 
after extended video block A. Audio track B' has 
also been moved such that its locked relationship 
with video block B is preserved. Audio track B* has 
pushed audio track D' to the right so that the two 
audio tracks don't overlap. Video block D which is 
locked to audio track D* has been moved to the 
right in order to preserve the locked relative rela- 
tionship with audio track D\ 

Bi-directional Domino Ripple 

All of the examples provided in this Specifica- 
tion have only illustrated re-edits which extend the 
out point of a video block forward in time and 
which ripple video blocks forward in time. However, 
the system of the present invention works equally 
well when an in point of a video block is moved 
backward in time and causing the video blocks to 
ripple backward in time. 

Figure 11 illustrates an example of a re-edit 
which causes a backward ripple. The first video 
time line illustrated in Figure 11 has four video 
blocks A, B, C, and D. In the second video time 
line illustrated in Figure 11, the in point of video 
block D has been move backward in time to extend 
past the out point of video block C. The third video 
time line of Figure 11 illustrates how the system 
reconciles the inconsistent video time line. In the 
third video time line of Figure 11, the system has 
moved video blocks A, B t and C, backward in time 
in order to make room for the extended video block 
D. Thus the system rippled video blocks B and C 
backward in time. 

Furthermore, the system of the present inven- 
tion can be configured to ripple media blocks both 
forward and backward (or backward and forward) 
as necessary. A forward and backward ripple is 
necessary when just a forward ripple or backward 
ripple alone would not reconcile the inconsistent 



time line. 

Figure 12 illustrates an example of a re-edit 
which causes both a forward and backward ripple. 
The first video time line illustrated in Figure 10 has 
5 five video blocks A, B, C, D, and E. Video block E 
is locked to the reference time line to prevent it 
from moving. In the second video time line illus- 
trated in Figure 12, the out point of video block B 
has been extended past the in point of video block 

10 C such that the system must reconcile the time 
line. The third video time line of Figure 12 illus- 
trates a first pass made by the system when at- 
tempting to reconcile the inconsistent video time 
line by rippling the video blocks forward. In the 

75 third video time line of Figure 12, the system has 
moved video blocks C and D after extended video 
block B. However, this has caused video block D to 
overlap video block E which is locked to the refer- 
ence time line. The system then attempts to ripple 

20 backwards. In the fourth video time line of Figure 
12, the system has moved video blocks A, B, C 
and D backward in time such that video blocks D 
and E no longer overlap. Thus the system was able 
to reconcile the re-edit without affecting locked 

25 video block E by rippling video blocks forward in 
time and then backward in time. 

If video block A in Figure 12 was locked to the 
reference time line; the second backward ripple 
pass would have failed since the system have not 

30 been able to move video block A over to the left. In 
such a situation, the system can handle the prob- 
lem in one of two ways. In a first embodiment of 
the present invention, the system handles such a 
problem by displaying a pop-up dialog box which 

35 explains the problem to the user. The dialog box 
would ask the user whether the edit should be 
aborted or whether video block A or E should be 
unlocked. In a second embodiment the present 
invention uses a set of predefined user preferences 

40 to handle the problem. For example, the user may 
specify a set of preferences that cause the system 
to always abort any re-edit which does not fit within 
the existing gaps. 

45 The Domino Delete Re-Edit Function 

The "Domino" effect can also be used when a 
user performs a re-edit which shortens the duration 
of video block. In such a situation, the system 
so moves nearby video blocks over such that the 
space created by the shortened video block is 
filled. 

Figure 13 illustrates an example of a re-edit 
which shortens the duration of a video block. The 
55 first video time line illustrated in Figure 13 has four 
video blocks A, B, C, and D. In the second video 
time line illustrated in Figure 13, the out point of 
video block B has been moved backward in time 
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such that video block B now has a shorter duration. 
By shortening video block B extra space has been 
created between video blocks B and C. The third 
video time line of Figure 13 illustrates how the 
system adjusts the remaining video blocks in order 
to fill in the extra space created by shortened video 
block B. In the third video time line of Figure 13, 
the system has moved video blocks C and D 
backward in time after shortened video block B. 
The system preserves the empty spaces that ex- 
isted before video block B was shortened. 

Although the present invention has been de- 
scribed in terms of a preferred embodiment, it will 
be appreciated that various modifications and alter- 
ations might be made by those skilled in the art 
without departing from the spirit and score of the 
invention. The invention should therefore be mea- 
sured in terms of the claims which follow. 

Claims 

1. A method of re-editing a multi-media work 
using a computer controlled multi-media re- 
editing system, said multi-media work compris- 
ing at least one media track laid on a reference 
time line, said media track comprising a plural- 
ity of media blocks laid on said reference time 
line, each of said media blocks comprising an 
in point and an out point, said media track 
having gaps between said media blocks, said 
method comprising the steps of: 

extending a first media block in a first 
direction such that a duration of said first me- 
dia block is increased, said extended first me- 
dia block overlapping a second media block 
located next to said first media block in said 
first direction; 

moving said second media block along 
said reference time line in said first direction 
such that said second media block is adjacent 
to said first media block and said second me- 
dia block no longer overlaps said first media 
block; and 

moving additional media blocks next to 
said second media block along said reference 
time line in said first direction while filling in 
said gaps between said media blocks such 
that a duration of gaps filled equals the in- 
crease in duration of said first media block and 
no media blocks overlap. 

2. The method of re-editing a multi-media work 
as claimed in claim 1 wherein said step of 
extending a first media block in a first direction 
comprises the substep of: 

extending the out point of said first media 
block forward along reference said time line 
such that said out point extends past the in 
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point of said second media block. 

3. The method of re-editing a multi-media work 
as claimed in claim 1 or 2 wherein said step of 

s extending a first media block in a first direction 

comprises the substep of: 

extending the in point of said first media 
block backward along reference said time line 
such that said in point extends past the out 

w point of said second media block. 

4. The method of re-editing a multi-media work 
as claimed in claim 1, 2 or 3 wherein said 
second media block is immediately adjacent to 

15 said first media block and step of moving 

additional media blocks only moves adjacent 
media blocks that form a contiguous set of 
media blocks. 

20 5. The method of re-editing a multi-media work 
as claimed in claim 1, 2, 3 or 4 wherein said 
media blocks can be locked to said reference 
time line such that either step of moving media 
blocks is aborted if either step attempts to 

25 move a locked media block. 

6. The method of re-editing a multi-media work 
as claimed fn claim 5 wherein said computer 
controlled re-editing system informs a user 

30 when either step of moving media blocks is 

aborted. 

7. The method of re-editing a multi-media work 
as claimed in claim 5 or 6 wherein said media 

35 blocks can be locked to said reference time 

line such that if either step of moving media 
blocks fails due to a locked media block then 
said system attempts to move said first, sec- 
ond, and additional video blocks in second 

40 direction opposite of said first direction. 

8- The method of re-editing a multi-media work 
as claimed in any one of the preceding claims 
wherein a third media block can be locked to a 
45 fourth media block such either step of moving 

media blocks moves said third and fourth me- 
dia blocks as a single unit such that the rela- 
tive relationship between said third and fourth 
media blocks is preserved. 

50 

9. The method of re-editing a multi-media work 
as claimed in claim 8 wherein said third media 
block and said fourth media block are from two 
different media tracks. 

55 

10. A method of re-editing a multi-media work 
using a computer controlled multi-media re- 
editing system, said multi-media work compris- 
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ing at least one media track laid on a reference 
time line, said media track comprising a plural- 
ity of media blocks laid on said reference time 
line, each of said media blocks comprising an 
in point and an out point, said media track 5 
having gaps between said media blocks, said 
method comprising the steps of: 

shortening a first media block such that a 
duration of said first media block is decreased, 
said shortened first media block creating an io 
empty space on said time line; 

moving additional media blocks next to 
said first media block closer to said first media 
block such that said empty space created by 
said shortened first media block is filled while 75 
preserving said gaps between said media 
blocks. 

11. A computer controlled multi-media re-editing 
apparatus for re-editing a multi-media work, 20 
<;ah1 multt-mftdia work comprising at least one 
m#rl»;i trar.K laid on a reference time line, said 
mel»a track comprising a plurality of media 
blocks laid on said reference time line, each of 

said meoa blocks comprising an in point and 25 
an out point, said media track having gaps 
between said media blocks, said apparatus 
comprising the elements of: 

moans for extending a first media block in 
a fi-st direction such that a duration of said first 30 
mo-Jia block is increased, said extended first 
media block overlapping a second media block 
located next to said first media block; 

moans for moving said second media 
block along said reference time line in said 35 
lirs: Direction such that said second media 
block is adjacent to said first media block and 
no longer overlaps said first media block; and 

means for moving additional media blocks 
along said reference time line in said first 40 
direction while filling in said gaps between said 
media blocks such that a duration of gaps 
filled equals the increase in duration of said 
firs: media block and no media blocks overlap. 

45 

12. The computer controlled multi-media re-editing 
apparatus for re-editing a multi-media work as 
claimed in claim 11 wherein said means for 
extending a first media block in a first direction 
extends the out point of said first media block 50 
forward along reference said time line such 

that said out point extends past the in point of 
said second media block. 

13. The computer controlled multi-media re-editing 55 
apparatus for re-editing a multi-media work as 
claimed in claim 11 or 12 wherein said means 

for extending a first media block in a first 



direction extends the in point of said first me- 
dia block backward along reference said time 
line such that said in point extends past the 
out point of said second media block; 

14. The computer controlled multi-media re-editing 
apparatus for re-editing a multi-media work as 
claimed in claim 11, 12 or 13 wherein said 
second media block is immediately adjacent to 
said first media block and means for moving 
additional media blocks only moves adjacent 
media blocks that form a contiguous set of 
media blocks. 

15. The computer controlled multi-media re-editing 
apparatus for re-editing a multi -media work as 
claimed in claim 11, 12, 13 or 14 wherein said 
media blocks can be locked to said reference 
time line such that said means for moving 
additional media blocks aborts when said 
means for moving additional media blocks at- 
tempts to move a media block locked to said 
reference time line. 

16. The computer controlled multi-media re-editing 
apparatus for re^editing a multi-media work as 
claimed in claim 15 wherein said computer 
controlled re^eeliting apparatus informs a user 
when means for moving additional media 
blocks attempts to move a media block locked 
to said reference time line. 

17. The computer controlled multi-media re-editing 
apparatus for re-editing a multi-media work as 
claimed in any one of claims 11 to 16 wherein 
said media blocks can be locked to said refer- 
ence time line such that if said means moving 
media blocks fails due to a locked media block 
then means moving media blocks attempts to 
move said first, second, and additional video 
blocks in second direction opposite of said first 
direction. 

18. The computer controlled multi-media re-editing 
apparatus for re-editing a multi-media work as 
claimed in any one of claims 11 to 17 wherein 
a third media block can be locked to a fourth 
media block and said means for moving media 
blocks moves said third and fourth media 
blocks as a single unit such that the relative 
relationship between said third and fourth me- 
dia blocks is preserved. 

19. The Computer controlled multi-media re-editing 
apparatus for re-editing a multi-media work as 
claimed in claim 18 wherein said third media 
block and said fourth media block are from two 
different media tracks. 



10 

CID: <EP . 0625782A2_l_> 



19 EP 0 625 782 A2 20 

20. A mufti-media work re-edited by a method 
according to any one of claims 1 to 10 or 
using an apparatus according to any one of 
claims 11 to 19. 
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@ A computer controlled video re-editing system 
with a "Domino" re-edit function. An edited video 
comprises a set of ordered video segments, in a 
video re-editing system a user can re-edit an edited 
video by adjusting the duration of each video seg- 
ment. When a user adjusts the length of a video 
segment, the Domino re-edit function examines how 
the re-edit to the edited video work affects nearby 
video segments in the edited video work. The 
Domino re-edit function then propagates the effects 
of a re-edit by moving the nearby video segments 
forward or backward along a time line to accom- 
modate the re-edit made. The Domino re-edit func- 
tion utilizes any empty spaces that exist in an edited 
video work such that the empty spaces are filled 
before the effects of a re-edit are propagated to 
another video segment, if there is sufficient empty 
space between video segments, the effects of the 
re-edit cease propagating. To prevent a video seg- 
ment from being moved by the Domino re-edit func- 
tion, a user can "lock" the video segment to a 
reference time line. To preserve a relative relation- 



ship between two video segments, a user can lock 
the two video segments together. 
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